[Fix](Top-N opt) evicting quering rowsets in prior to correct use_count by eldenmoon · Pull Request #30904 · apache/doris

eldenmoon · 2024-02-06T08:19:45Z

This addresses the scenario where a rowset cannot be removed.

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-02-06T08:19:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

eldenmoon · 2024-02-06T08:20:10Z

run buildall

github-actions · 2024-02-06T08:29:40Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-02-06T08:55:36Z

TPC-H: Total hot run time: 37484 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false

------ Round 1 ----------------------------------
q1	18143	4725	4475	4475
q2	2202	148	137	137
q3	11331	955	961	955
q4	5052	844	737	737
q5	7915	2915	2918	2915
q6	189	124	127	124
q7	1177	773	763	763
q8	9521	2049	2056	2049
q9	7487	6396	6347	6347
q10	8104	2491	2402	2402
q11	413	210	205	205
q12	753	278	273	273
q13	18025	3310	3343	3310
q14	279	248	255	248
q15	523	494	497	494
q16	474	404	409	404
q17	947	494	520	494
q18	6924	6071	6054	6054
q19	1578	1488	1399	1399
q20	629	364	357	357
q21	6982	3159	3049	3049
q22	808	301	293	293
Total cold run time: 109456 ms
Total hot run time: 37484 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4460	4432	4465	4432
q2	323	245	230	230
q3	3005	2865	2804	2804
q4	1815	1598	1601	1598
q5	5198	5282	5249	5249
q6	191	114	116	114
q7	2176	1784	1728	1728
q8	3139	3277	3301	3277
q9	8314	8377	8354	8354
q10	5812	3546	3550	3546
q11	544	464	459	459
q12	752	586	576	576
q13	6657	3094	3113	3094
q14	264	253	260	253
q15	521	493	491	491
q16	496	479	478	478
q17	1861	1693	1728	1693
q18	8003	7779	7542	7542
q19	9014	1536	1544	1536
q20	2119	1921	1908	1908
q21	4792	4554	4614	4554
q22	540	471	506	471
Total cold run time: 69996 ms
Total hot run time: 54387 ms

doris-robot · 2024-02-06T09:06:58Z

TPC-DS: Total hot run time: 174687 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false

query1	942	355	337	337
query2	6505	2093	1973	1973
query3	6699	214	206	206
query4	31611	22009	22034	22009
query5	4322	431	414	414
query6	260	163	164	163
query7	4607	287	284	284
query8	243	175	180	175
query9	9255	2290	2281	2281
query10	427	236	232	232
query11	18074	15340	15493	15340
query12	131	77	75	75
query13	1621	411	410	410
query14	9296	7084	7118	7084
query15	262	174	185	174
query16	8138	259	248	248
query17	2091	527	495	495
query18	2104	264	258	258
query19	353	144	141	141
query20	82	82	81	81
query21	201	130	119	119
query22	4946	4678	4680	4678
query23	30763	30035	29976	29976
query24	10644	2731	2778	2731
query25	577	348	346	346
query26	1396	149	151	149
query27	2917	302	311	302
query28	7342	1858	1838	1838
query29	869	618	615	615
query30	285	130	139	130
query31	888	701	729	701
query32	94	53	51	51
query33	735	225	220	220
query34	1091	464	455	455
query35	875	778	775	775
query36	966	959	926	926
query37	121	56	56	56
query38	3195	3138	3121	3121
query39	1296	1252	1233	1233
query40	190	96	91	91
query41	39	34	34	34
query42	103	94	96	94
query43	533	491	503	491
query44	1029	688	712	688
query45	195	181	175	175
query46	1047	663	659	659
query47	1583	1477	1442	1442
query48	411	351	347	347
query49	1106	285	280	280
query50	750	370	387	370
query51	5236	5119	5143	5119
query52	100	87	83	83
query53	337	291	271	271
query54	265	215	216	215
query55	81	78	76	76
query56	219	194	209	194
query57	988	914	916	914
query58	194	176	188	176
query59	2424	2317	2445	2317
query60	236	218	209	209
query61	83	82	89	82
query62	641	378	399	378
query63	294	272	281	272
query64	5528	3687	3481	3481
query65	3204	3214	3218	3214
query66	1009	314	312	312
query67	14174	14314	14104	14104
query68	4302	549	529	529
query69	457	333	331	331
query70	1320	1239	1234	1234
query71	317	248	239	239
query72	5962	2801	2639	2639
query73	697	336	326	326
query74	6631	6237	6246	6237
query75	3002	2315	2304	2304
query76	2545	932	924	924
query77	338	234	225	225
query78	9110	8727	8453	8453
query79	3510	498	495	495
query80	2194	356	341	341
query81	535	196	205	196
query82	1539	82	93	82
query83	358	132	122	122
query84	283	80	79	79
query85	1609	346	329	329
query86	464	283	303	283
query87	3362	3224	3200	3200
query88	4388	2340	2325	2325
query89	441	366	367	366
query90	2003	163	166	163
query91	154	141	124	124
query92	60	44	41	41
query93	4216	508	459	459
query94	1320	172	175	172
query95	7957	360	354	354
query96	606	266	265	265
query97	4241	4110	4121	4110
query98	228	198	196	196
query99	1261	654	715	654
Total cold run time: 293633 ms
Total hot run time: 174687 ms

doris-robot · 2024-02-06T09:12:25Z

ClickBench: Total hot run time: 31.15 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.67	0.10	0.10
query5	0.52	0.52	0.52
query6	1.18	0.67	0.65
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.55	0.51	0.51
query10	0.55	0.54	0.55
query11	0.12	0.09	0.08
query12	0.10	0.08	0.09
query13	0.61	0.61	0.60
query14	0.79	0.81	0.81
query15	0.78	0.78	0.76
query16	0.40	0.37	0.38
query17	1.01	1.01	1.00
query18	0.22	0.26	0.24
query19	1.80	1.80	1.80
query20	0.01	0.01	0.01
query21	15.40	0.57	0.57
query22	2.54	2.79	1.99
query23	17.33	0.83	0.78
query24	2.39	0.71	0.90
query25	0.36	0.22	0.14
query26	0.53	0.14	0.14
query27	0.06	0.05	0.04
query28	12.35	0.84	0.86
query29	12.52	3.35	3.27
query30	0.57	0.57	0.54
query31	2.78	0.34	0.34
query32	3.38	0.47	0.48
query33	3.22	3.22	3.23
query34	15.81	4.28	4.27
query35	4.31	4.29	4.30
query36	1.08	1.07	1.07
query37	0.07	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.71 s
Total hot run time: 31.15 s

doris-robot · 2024-02-06T09:15:30Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.7 seconds inserted 10000000 Rows, about 729K ops/s

doris-robot · 2024-02-06T09:36:16Z

TeamCity be ut coverage result:
Function Coverage: 36.20% (8629/23839)
Line Coverage: 28.21% (70560/250126)
Region Coverage: 27.22% (36424/133791)
Branch Coverage: 24.00% (18656/77722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/eeaf78b488f96c522aa3645952f14cd32486f61b_eeaf78b488f96c522aa3645952f14cd32486f61b/report/index.html

…nt (apache#102) This addresses the scenario where a rowset cannot be removed.

eldenmoon · 2024-02-06T12:12:28Z

run buildall

github-actions · 2024-02-06T12:22:03Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-02-06T13:03:29Z

TPC-H: Total hot run time: 37066 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false

------ Round 1 ----------------------------------
q1	17686	4593	4464	4464
q2	2049	153	136	136
q3	10580	952	915	915
q4	4652	750	755	750
q5	7718	2792	2833	2792
q6	185	123	117	117
q7	1157	730	719	719
q8	9295	2025	2054	2025
q9	7313	6350	6378	6350
q10	8135	2429	2422	2422
q11	420	212	194	194
q12	757	266	291	266
q13	18014	3336	3322	3322
q14	279	263	254	254
q15	531	503	506	503
q16	491	408	417	408
q17	951	586	531	531
q18	6782	5965	5912	5912
q19	1609	1338	1304	1304
q20	569	362	353	353
q21	6897	3167	3033	3033
q22	809	310	296	296
Total cold run time: 106879 ms
Total hot run time: 37066 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4432	4453	4429	4429
q2	331	237	229	229
q3	2983	2861	2814	2814
q4	1887	1625	1630	1625
q5	5209	5228	5272	5228
q6	186	114	114	114
q7	2173	1784	1739	1739
q8	3118	3258	3249	3249
q9	8387	8309	8324	8309
q10	5921	3593	3576	3576
q11	543	478	486	478
q12	721	576	542	542
q13	16920	3097	3059	3059
q14	290	262	267	262
q15	544	495	489	489
q16	518	480	482	480
q17	1858	1689	1682	1682
q18	8048	7814	7515	7515
q19	6675	1553	1568	1553
q20	2140	1917	1895	1895
q21	4737	4590	4652	4590
q22	526	456	493	456
Total cold run time: 78147 ms
Total hot run time: 54313 ms

doris-robot · 2024-02-06T13:14:26Z

TPC-DS: Total hot run time: 174371 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false

query1	928	344	329	329
query2	6561	1988	1893	1893
query3	6708	205	200	200
query4	31715	22051	21992	21992
query5	4228	374	425	374
query6	269	171	170	170
query7	4603	305	285	285
query8	255	174	173	173
query9	8937	2288	2253	2253
query10	414	243	241	241
query11	18597	15509	15289	15289
query12	131	74	73	73
query13	1619	411	415	411
query14	9763	7049	6937	6937
query15	251	173	189	173
query16	8079	267	247	247
query17	1854	552	500	500
query18	2106	268	268	268
query19	191	142	138	138
query20	85	74	79	74
query21	203	129	112	112
query22	4898	4658	4584	4584
query23	30746	30092	30118	30092
query24	6139	2759	2778	2759
query25	484	339	335	335
query26	706	146	155	146
query27	2013	307	302	302
query28	3585	1864	1839	1839
query29	863	621	582	582
query30	206	134	142	134
query31	906	711	687	687
query32	93	54	54	54
query33	456	221	221	221
query34	816	460	462	460
query35	850	781	763	763
query36	1019	945	918	918
query37	94	56	57	56
query38	3298	3118	3090	3090
query39	1314	1272	1244	1244
query40	181	93	91	91
query41	38	36	34	34
query42	94	89	94	89
query43	483	489	485	485
query44	1040	683	694	683
query45	190	179	174	174
query46	1035	678	648	648
query47	1613	1490	1540	1490
query48	431	359	355	355
query49	620	296	282	282
query50	777	382	372	372
query51	5251	5123	5176	5123
query52	94	87	81	81
query53	328	267	269	267
query54	248	212	211	211
query55	81	77	75	75
query56	227	200	200	200
query57	973	875	934	875
query58	200	174	183	174
query59	2518	2229	2226	2226
query60	240	207	211	207
query61	85	84	86	84
query62	523	347	341	341
query63	297	260	273	260
query64	4253	3564	3674	3564
query65	3275	3217	3225	3217
query66	799	318	300	300
query67	14536	13998	14053	13998
query68	3915	547	546	546
query69	466	325	321	321
query70	1253	1218	1219	1218
query71	319	248	242	242
query72	5782	2837	2665	2665
query73	722	345	335	335
query74	6515	6330	6214	6214
query75	3010	2313	2359	2313
query76	2575	940	887	887
query77	344	228	221	221
query78	9174	8851	8706	8706
query79	2587	512	491	491
query80	1502	360	336	336
query81	504	196	196	196
query82	647	85	78	78
query83	244	118	126	118
query84	248	81	84	81
query85	1679	331	326	326
query86	493	296	289	289
query87	3444	3197	3171	3171
query88	3813	2374	2369	2369
query89	429	349	360	349
query90	1967	166	166	166
query91	154	120	122	120
query92	59	43	43	43
query93	2754	475	463	463
query94	1287	172	172	172
query95	487	368	352	352
query96	590	288	275	275
query97	4237	4125	4115	4115
query98	229	194	189	189
query99	1133	700	722	700
Total cold run time: 268975 ms
Total hot run time: 174371 ms

doris-robot · 2024-02-06T13:19:54Z

ClickBench: Total hot run time: 31.34 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.66	0.09	0.09
query5	0.55	0.52	0.51
query6	1.18	0.65	0.63
query7	0.01	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.49	0.48
query10	0.55	0.55	0.56
query11	0.13	0.09	0.09
query12	0.12	0.09	0.09
query13	0.61	0.60	0.61
query14	0.78	0.80	0.78
query15	0.80	0.79	0.79
query16	0.40	0.38	0.39
query17	1.05	0.98	1.02
query18	0.23	0.28	0.22
query19	1.89	1.79	1.80
query20	0.02	0.01	0.01
query21	15.40	0.56	0.55
query22	2.56	2.97	2.50
query23	17.26	0.84	0.81
query24	2.61	0.81	0.47
query25	0.30	0.13	0.19
query26	0.37	0.14	0.14
query27	0.06	0.05	0.05
query28	12.09	0.91	0.86
query29	12.51	3.26	3.29
query30	0.64	0.55	0.53
query31	2.78	0.34	0.35
query32	3.35	0.48	0.48
query33	3.21	3.24	3.21
query34	15.84	4.29	4.29
query35	4.31	4.34	4.28
query36	1.09	1.06	1.06
query37	0.07	0.05	0.05
query38	0.04	0.03	0.02
query39	0.02	0.01	0.02
query40	0.16	0.12	0.14
query41	0.08	0.02	0.01
query42	0.03	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 105.72 s
Total hot run time: 31.34 s

doris-robot · 2024-02-06T13:23:01Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

doris-robot · 2024-02-06T13:26:41Z

TeamCity be ut coverage result:
Function Coverage: 36.19% (8637/23863)
Line Coverage: 28.21% (70620/250369)
Region Coverage: 27.21% (36441/133927)
Branch Coverage: 23.99% (18666/77814)
Coverage Report: http://coverage.selectdb-in.cc/coverage/32a23524fe4a99f1a696776ed3ea5e9a0e78027a_32a23524fe4a99f1a696776ed3ea5e9a0e78027a/report/index.html

github-actions · 2024-02-07T02:59:47Z

clang-tidy review says "All clean, LGTM! 👍"

xiaokang

LGTM

github-actions · 2024-02-07T03:17:54Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-02-07T03:17:56Z

PR approved by anyone and no changes requested.

xiaokang · 2024-02-08T05:20:47Z

run buildall

doris-robot · 2024-02-08T05:41:29Z

TPC-H: Total hot run time: 41474 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false

------ Round 1 ----------------------------------
q1	17660	5018	4905	4905
q2	2040	147	140	140
q3	10570	1041	1000	1000
q4	4658	978	970	970
q5	7711	3225	3268	3225
q6	197	136	133	133
q7	1276	774	757	757
q8	9262	2073	2084	2073
q9	7510	6720	6694	6694
q10	8316	2638	2664	2638
q11	422	214	205	205
q12	754	333	329	329
q13	17988	3677	3673	3673
q14	294	268	252	252
q15	596	508	523	508
q16	471	402	421	402
q17	922	874	821	821
q18	7456	6715	6636	6636
q19	1526	1497	1516	1497
q20	621	365	312	312
q21	6996	4016	3972	3972
q22	873	344	332	332
Total cold run time: 108119 ms
Total hot run time: 41474 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4894	4829	4875	4829
q2	293	184	187	184
q3	3616	3602	3598	3598
q4	2557	2556	2537	2537
q5	5776	5753	5746	5746
q6	211	129	127	127
q7	2255	1682	1687	1682
q8	3035	3125	3111	3111
q9	8766	8777	8726	8726
q10	6780	4251	4252	4251
q11	533	378	383	378
q12	761	542	536	536
q13	5110	3409	3432	3409
q14	279	235	232	232
q15	624	501	523	501
q16	504	437	422	422
q17	2008	1632	1616	1616
q18	8341	7817	7769	7769
q19	1627	1637	1638	1637
q20	2116	1826	1814	1814
q21	6598	6170	6169	6169
q22	561	500	507	500
Total cold run time: 67245 ms
Total hot run time: 59774 ms

doris-robot · 2024-02-08T05:52:50Z

TPC-DS: Total hot run time: 180875 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false

query1	943	361	352	352
query2	6511	1753	1800	1753
query3	6698	207	205	205
query4	24333	22620	22537	22537
query5	4303	469	467	467
query6	262	167	165	165
query7	4610	315	289	289
query8	256	194	201	194
query9	8461	2848	2848	2848
query10	417	232	218	218
query11	16259	15703	15574	15574
query12	134	81	83	81
query13	1682	429	409	409
query14	9018	7735	7683	7683
query15	219	184	190	184
query16	7343	250	249	249
query17	1410	570	539	539
query18	1957	269	266	266
query19	197	146	157	146
query20	84	82	83	82
query21	190	125	110	110
query22	5072	4796	4863	4796
query23	32536	31532	31596	31532
query24	12701	3399	3443	3399
query25	652	378	369	369
query26	1880	155	159	155
query27	3053	316	315	315
query28	6656	1829	1835	1829
query29	1180	622	620	620
query30	277	142	146	142
query31	977	781	782	781
query32	90	60	58	58
query33	719	232	243	232
query34	1058	483	499	483
query35	946	831	844	831
query36	993	901	869	869
query37	157	55	67	55
query38	3453	3349	3351	3349
query39	1372	1341	1330	1330
query40	304	104	106	104
query41	37	35	33	33
query42	110	95	101	95
query43	473	448	448	448
query44	1064	698	707	698
query45	203	182	173	173
query46	1045	788	759	759
query47	1764	1597	1592	1592
query48	423	344	340	340
query49	1197	311	302	302
query50	771	395	373	373
query51	5325	5213	5172	5172
query52	109	91	91	91
query53	390	294	301	294
query54	291	221	219	219
query55	88	76	80	76
query56	220	212	196	196
query57	1043	976	995	976
query58	215	197	195	195
query59	2273	2289	2141	2141
query60	234	214	216	214
query61	87	87	83	83
query62	603	402	390	390
query63	328	284	281	281
query64	6504	3076	3174	3076
query65	3318	3258	3224	3224
query66	1364	326	324	324
query67	14520	14122	14319	14122
query68	5103	516	540	516
query69	519	361	359	359
query70	1304	1212	1201	1201
query71	377	255	254	254
query72	6334	2745	2614	2614
query73	702	305	307	305
query74	7061	6669	6713	6669
query75	3197	2550	2557	2550
query76	3210	1161	1209	1161
query77	346	237	228	228
query78	9446	8761	8753	8753
query79	968	499	493	493
query80	516	354	346	346
query81	402	206	216	206
query82	165	83	82	82
query83	141	123	125	123
query84	224	79	80	79
query85	755	344	345	344
query86	295	319	302	302
query87	3607	3438	3405	3405
query88	2690	2278	2275	2275
query89	440	353	352	352
query90	1921	165	166	165
query91	154	126	130	126
query92	54	49	48	48
query93	1029	520	476	476
query94	1061	178	183	178
query95	482	369	8651	369
query96	583	262	260	260
query97	4499	4312	4255	4255
query98	216	206	197	197
query99	1060	750	697	697
Total cold run time: 272223 ms
Total hot run time: 180875 ms

doris-robot · 2024-02-08T05:58:13Z

ClickBench: Total hot run time: 30.68 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.02
query3	0.22	0.07	0.07
query4	1.64	0.09	0.08
query5	0.48	0.48	0.48
query6	1.35	0.66	0.62
query7	0.02	0.01	0.02
query8	0.04	0.02	0.02
query9	0.53	0.44	0.44
query10	0.50	0.48	0.47
query11	0.12	0.10	0.09
query12	0.12	0.10	0.10
query13	0.59	0.58	0.58
query14	0.76	0.78	0.78
query15	0.82	0.79	0.79
query16	0.32	0.33	0.33
query17	0.90	0.88	0.86
query18	0.17	0.17	0.17
query19	1.72	1.67	1.68
query20	0.01	0.02	0.01
query21	15.40	0.62	0.61
query22	3.26	3.37	2.28
query23	17.90	1.04	1.00
query24	2.00	0.38	0.22
query25	0.58	0.06	0.06
query26	0.15	0.14	0.14
query27	0.05	0.06	0.05
query28	12.38	0.84	0.80
query29	12.55	3.35	3.34
query30	0.52	0.48	0.47
query31	2.78	0.36	0.37
query32	3.33	0.48	0.47
query33	3.16	3.06	3.19
query34	15.38	4.49	4.51
query35	4.49	4.50	4.50
query36	1.07	0.96	0.94
query37	0.07	0.05	0.06
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.18	0.15	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 105.82 s
Total hot run time: 30.68 s

doris-robot · 2024-02-08T05:59:30Z

TeamCity be ut coverage result:
Function Coverage: 35.86% (8556/23862)
Line Coverage: 27.78% (69458/249999)
Region Coverage: 26.92% (36039/133874)
Branch Coverage: 23.70% (18425/77754)
Coverage Report: http://coverage.selectdb-in.cc/coverage/59c54f323b6f549e66b88e51169775c58c89fca4_59c54f323b6f549e66b88e51169775c58c89fca4/report/index.html

doris-robot · 2024-02-08T06:01:18Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

…nt (#102) (#30904) This addresses the scenario where a rowset cannot be removed.

…nt (apache#102) (apache#30904) This addresses the scenario where a rowset cannot be removed.

xiaokang added usercase Important user case type label dev/2.0.5 labels Feb 6, 2024

[Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou…

32a2352

…nt (apache#102) This addresses the scenario where a rowset cannot be removed.

eldenmoon force-pushed the fix-rs-cnt branch from eeaf78b to 32a2352 Compare February 6, 2024 12:12

Merge branch 'master' into fix-rs-cnt

59c54f3

xiaokang added dev/2.0.5-merged and removed dev/2.0.5 labels Feb 7, 2024

xiaokang changed the title ~~[Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou…~~ [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count Feb 7, 2024

xiaokang approved these changes Feb 7, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 7, 2024

github-actions bot added the reviewed label Feb 7, 2024

yiguolei approved these changes Feb 8, 2024

View reviewed changes

yiguolei merged commit 645f6d4 into apache:master Feb 8, 2024

yiguolei pushed a commit that referenced this pull request Feb 16, 2024

[Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou…

eaaab33

…nt (#102) (#30904) This addresses the scenario where a rowset cannot be removed.

mymeiyi pushed a commit to mymeiyi/doris that referenced this pull request Feb 19, 2024

[Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou…

04e1b34

…nt (apache#102) (apache#30904) This addresses the scenario where a rowset cannot be removed.

TsukiokaKogane mentioned this pull request Mar 8, 2024

[Fix](rowset) When a rowset is cooled down, it is directly deleted. This can result in data query misses in the second phase of a two-phase query. #21741

Merged

Conversation

eldenmoon commented Feb 6, 2024 • edited by xiaokang Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

eldenmoon commented Feb 6, 2024

Uh oh!

github-actions bot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

eldenmoon commented Feb 6, 2024

Uh oh!

github-actions bot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

doris-robot commented Feb 6, 2024

Uh oh!

github-actions bot commented Feb 7, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 7, 2024

Uh oh!

github-actions bot commented Feb 7, 2024

Uh oh!

xiaokang commented Feb 8, 2024

Uh oh!

doris-robot commented Feb 8, 2024

Uh oh!

doris-robot commented Feb 8, 2024

Uh oh!

doris-robot commented Feb 8, 2024

Uh oh!

doris-robot commented Feb 8, 2024

Uh oh!

doris-robot commented Feb 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

eldenmoon commented Feb 6, 2024 •

edited by xiaokang

Loading